Papers with RL training
DeepTrans: Deep Reasoning Translation via Reinforcement Learning (2026.tacl-1)
Copied to clipboard
| Challenge: | Recent studies have shown promising performance in various downstream tasks. |
| Approach: | They propose a deep reasoning translation model that learns free translation via reinforcement learning (RL) they build a reward model with pre-defined scoring criteria on the translation results and thought processes . |
| Outcome: | The proposed model outperforms strong deep reasoning LLMs in literature translation and outperformed other models. |
Learning Diverse Responses with Prefix-Conditioned Supervised Fine-Tuning (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models exhibit highly homogeneous, repetitive responses, resulting in inefficient exploration. |
| Approach: | They propose a method that constructs semantically consistent yet distributionally distinct prior contents to different responses and decouple the one-to-many mapping. |
| Outcome: | The proposed method improves absolute performance by 5.3% and increases generation diversity by 198.3% on average while significantly enhancing output diversity and test-time scaling. |
ScaleBox: Enabling High-Fidelity and Scalable Code Verification for Large Language Models (2026.acl-demo)
Copied to clipboard
Jiasheng Zheng, Xin Zheng, Boxi Cao, Pengbo Wang, Zhengzhao Ma, Qiming Zhu, Jiazhen Jiang, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun
| Challenge: | Existing code sandboxes fail to provide accurate verification and efficiency under high-concurrency workloads. |
| Approach: | They propose a high-fidelity code verification system that provides sandbox feedback for RL training and evaluation. |
| Outcome: | The proposed system outperforms heuristic-matching baselines on LiveCodeBench and training stability on high-concurrency workloads. |
Nemotron-CrossThink: Scaling Self-Learning beyond Math Reasoning (2026.eacl-long)
Copied to clipboard
Syeda Nahida Akter, Shrimai Prabhumoye, Matvei Novikov, Seungju Han, Ying Lin, Evelina Bakhturina, Eric Nyberg, Yejin Choi, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro
| Challenge: | Prior work has successfully applied Reinforcement Learning (RL) to mathematical reasoning, but generalization to broader domains remains challenging due to limited data and lack of verifiable rewards for unstructured domains. |
| Approach: | They propose a framework that integrates multi-domain corpora into RL training to improve generalization across diverse reasoning tasks. |
| Outcome: | The proposed framework improves generalization across diverse reasoning tasks. |
Enhancing Efficiency and Exploration in Reinforcement Learning for LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches allocate an equal number of rollouts to all questions during the RL process, which is inefficient. |
| Approach: | They propose a mechanism for dynamically allocating rollout budgets based on the difficulty of the problems, enabling more efficient RL training. |
| Outcome: | The proposed model improves response precision while preserving exploratory ability to uncover potential correct pathways. |
TemplateRL: Structured Template-Guided Reinforcement Learning for LLM Reasoning (2026.findings-acl)
Copied to clipboard
Jinyang Wu, Chonghua Liao, Mingkuan Feng, Shuai Zhang, Zhengqi Wen, Haoran Luo, Ling Yang, Huazhe Xu, Jianhua Tao
| Challenge: | Existing RL methods rely on unstructured self-sampling to fit scalar rewards, resulting in inefficient rollouts. |
| Approach: | They propose a structured template-guided RL framework that augments policy optimization with explicit template guidance. |
| Outcome: | Experiments show that TemplateRL outperforms GRPO and GRPI by 99% on AIME and 41% on AMC with superior stability on weak models and remarkable cross-domain generalization. |
Entropy Scheduling in Reinforcement Learning for Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | entropy in reinforcement learning functions analogously to the learning rate in LLMs. |
| Approach: | They propose an entropy scheduling system that optimizes different pre-set goals by controlling and scheduling entropicy at each step of the RL process. |
| Outcome: | The proposed method improves AIME2024 from 50.9 to 54.9 within 40 training steps. |
StepCoder: Improving Code Generation with Reinforcement Learning from Compiler Feedback (2024.acl-long)
Copied to clipboard
Shihan Dou, Yan Liu, Haoxiang Jia, Enyu Zhou, Limao Xiong, Junjie Shan, Caishuang Huang, Xiao Wang, Xiaoran Fan, Zhiheng Xi, Yuhao Zhou, Tao Ji, Rui Zheng, Qi Zhang, Tao Gui, Xuanjing Huang
| Challenge: | Existing work integrates reinforcement learning with compiler feedback to enhance code generation quality but the long code generated by LLMs makes RL exploration ineffective. |
| Approach: | They propose a framework that integrates reinforcement learning and compiler feedback to enhance code generation quality. |
| Outcome: | The proposed framework outperforms state-of-the-art approaches in corresponding benchmarks and integrates reinforcement learning with compiler feedback to improve code generation quality. |
RED: Unleashing Token-Level Rewards from Holistic Feedback via Reward Redistribution (2025.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results demonstrate the superiority of our approach to aligning large language models with human preferences. |
| Approach: | They propose a method that evaluates and assigns specific credit to each token using an off-the-shelf reward model. |
| Outcome: | The proposed method evaluates and assigns specific credit to each token using an off-the-shelf reward model. |
Reward Gaming in Conditional Text Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Recent work has used reward functions learned from human annotations to align conditional text generation models with desired behaviors. |
| Approach: | They propose to use reinforcement learning to train conditional text generation models with reward functions learned from human annotations to align outputs with desired behaviors. |
| Outcome: | The proposed framework improves the quality of generated summaries by using saliency and faithfulness metrics. |
Neuro-Symbolic Reinforcement Learning with First-Order Logic (2021.emnlp-main)
Copied to clipboard
Daiki Kimura, Masaki Ono, Subhajit Chaudhury, Ryosuke Kohita, Akifumi Wachi, Don Joven Agravante, Michiaki Tatsubori, Asim Munawar, Alexander Gray
| Challenge: | Existing deep reinforcement learning methods require many trials before convergence and no direct interpretability of trained policies is provided. |
| Approach: | They propose a novel RL method which can learn symbolic and interpretable rules in their differentiable network. |
| Outcome: | The proposed method can learn symbolic and interpretable rules in their differentiable network. |
Mid-Think: Training-Free Intermediate-Budget Reasoning via Token-Level Triggers (2026.findings-acl)
Copied to clipboard
Van Yang, Shouren Wang, Debargha Ganguly, Xinpeng Li, Chaoda Song, Vikash Singh, Vipin Chaudhary, Xiaotian Han
| Challenge: | Explicit /think> tags are used to expose intermediate reasoning and enable hybrid thinking behaviors. |
| Approach: | They propose a training-free prompting format that combines these triggers to achieve intermediate-budget reasoning, outperforming fixed-token and prompt-based baselines in terms of the accuracy–length trade-off. |
| Outcome: | The proposed method outperforms fixed-token and prompt-based prompts in accuracy–length trade-offs while improving Qwen3-8B on AIME from 69.8% to 72.4% and on GPQA from 58.5% to 61.1%. |
Progra: Progress-Aware Reinforcement Learning for Multi-Turn Function Calling (2026.findings-acl)
Copied to clipboard
Huacan Chai, Zijie Cao, Maolin Ran, Yingxuan Yang, Jianghao Lin, Xin Peng, Hairui Wang, Renjie Ding, Ziyu Wan, Muning Wen, Weiwen Liu, Weinan Zhang, Fei Huang, Ying Wen
| Challenge: | Existing methods for multi-turn function calling are limited by redundancy and lack explicit integration of progress awareness into training. |
| Approach: | They propose a framework that explicitly integrates progress awareness into LLM training for multi-turn function calling. |
| Outcome: | Empirical results show that Progra outperforms existing methods on two public benchmarks. |
Improving Large Language Models via Fine-grained Reinforcement Learning with Minimum Editing Constraint (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing reinforcement learning methods do not provide fine-grained supervision for complex reasoning tasks. |
| Approach: | They propose a reinforcement learning method that incorporates a generative model as the reward model and a token-level supervision model for RL training. |
| Outcome: | Experiments on 8 tasks show the proposed method is effective . |
InternLM-XComposer2.5-Reward: A Simple Yet Effective Multi-Modal Reward Model (2025.findings-acl)
Copied to clipboard
Yuhang Zang, Xiaoyi Dong, Pan Zhang, Yuhang Cao, Ziyu Liu, Shengyuan Ding, Shenxi Wu, Yubo Ma, Haodong Duan, Wenwei Zhang, Kai Chen, Dahua Lin, Jiaqi Wang
| Challenge: | Despite the promising performance of Large Vision Language Models, they sometimes generate incorrect outputs. |
| Approach: | They propose a multi-modal reward model that aligns LVLMs with human preferences. |
| Outcome: | The proposed model achieves excellent results on the latest multi-modal reward model benchmark and shows competitive performance on text-only reward model. |
Reinforcement Learning for Aligning Large Language Models Agents with Interactive Environments: Quantifying and Mitigating Prompt Overfitting (2025.findings-naacl)
Copied to clipboard
Mohamed Salim Aissi, Clément Romac, Thomas Carta, Sylvain Lamprier, Pierre-Yves Oudeyer, Olivier Sigaud, Laure Soulier, Nicolas Thome
| Challenge: | Reinforcement learning (RL) is a promising approach for aligning large language models knowledge with sequential decision-making tasks. |
| Approach: | They propose to use a contrastive loss framework to analyze the sensitivity of LLMs to prompt formulations following RL training in a textual environment. |
| Outcome: | The proposed framework improves the model's robustness and generalization capabilities by minimizing the model’s internal representations and salient tokens. |
A Study of Reinforcement Learning for Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Recent studies have shown that reinforcement learning (RL) is an effective approach for improving the performance of neural machine translation systems. |
| Approach: | They propose to leverage reinforcement learning to boost the performance of NMT systems trained with monolingual data. |
| Outcome: | The proposed method achieves competitive results on translation tasks in English-German, Chinese-English and English-English systems. |
Selective Token Generation for Few-shot Natural Language Generation (2022.coling-1)
Copied to clipboard
| Challenge: | Experimental results show that the proposed selective token generation algorithm outperforms the previous additive learning algorithms based on the PLMs. |
| Approach: | They propose an additive learning algorithm that selectively outputs language tokens between a task-general PLM and a specific adapter during training and inference. |
| Outcome: | The proposed algorithm outperforms existing methods on few-shot natural language generation tasks. |
Enhancing Reinforcement Learning with Dense Rewards from Language Model Critic (2024.emnlp-main)
Copied to clipboard
| Challenge: | Reinforcement learning (RL) can align language models with non-differentiable reward signals, such as human preferences, but the sparsity of these signals can lead to inefficient and unstable learning. |
| Approach: | They propose a framework that utilizes the critique capability of Large Language Models to produce intermediate-step rewards during RL training. |
| Outcome: | The proposed framework improves sample efficiency and the overall performance of the policy model, supported by both automatic and human evaluation. |
ACECODER: Acing Coder RL via Automated Test-Case Synthesis (2025.acl-long)
Copied to clipboard
| Challenge: | Recent coder models have been driven by supervised fine-tuning, but the potential of reinforcement learning remains unexplored due to the lack of reliable reward data/model in the code domain. |
| Approach: | They propose a pipeline that generates extensive test-case pairs from existing code data and constructs preference pairs based on pass rates over sampled programs. |
| Outcome: | The proposed pipeline generates extensive (question, test-cases) pairs from existing code data and trains them with Bradley-Terry loss. |
TreeRL: LLM Reinforcement Learning with On-Policy Tree Search (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for On-Policy LLM RL typically train a separate process reward model, which suffers from distribution mismatch and reward hacking. |
| Approach: | They propose a reinforcement learning framework that directly incorporates on-policy tree search for RL training. |
| Outcome: | Experiments on math and code reasoning benchmarks show that tree search achieves superior performance compared to traditional ChainRL. |
SPS: Steering Probability Squeezing for Better Exploration in Reinforcement Learning for Large Language Models (2026.findings-acl)
Copied to clipboard
Yifu Huo, Chenglong Wang, Ziming Zhu, Shunjie Xing, Peinan Feng, Tongran Liu, Qiaozhi He, Tian Hua Zhou, null Changxiaojia, JingBo Zhu, Zhengtao Yu, Tong Xiao
| Challenge: | Reinforcement learning (RL) training typically improves single-sample success rates but limited exploration of diverse reasoning trajectories. |
| Approach: | They propose a training paradigm that interleaves conventional RL with inverse reinforcement learning (IRL) they propose 'Steering Probability Squeezing' to enhance exploration without external supervision . |
| Outcome: | The proposed training paradigm improves Pass@k and improves exploration of diverse reasoning trajectories without external supervision. |
Reinforcement Replaces Supervision: Query focused Summarization using Deep Reinforcement Learning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Query-focused Summarization (QfS) is a system that generates summaries from document(s) based on a query. |
| Approach: | They propose a Query-focused Summarization approach that uses a generalization of Reinforcement Learning (RL) for Natural Language Generation and a better semantic similarity reward. |
| Outcome: | The proposed approach improves on the ROUGE-L metric and in a benchmark dataset. |
Table-R1: Inference-Time Scaling for Table Reasoning Tasks (2025.emnlp-main)
Copied to clipboard
| Challenge: | In this study, we explore inference-time scaling on table reasoning tasks. |
| Approach: | They propose a large-scale dataset of reasoning traces and a reinforcement learning with verifiable rewards approach to enable inference-time scaling on table reasoning tasks. |
| Outcome: | The proposed model matches or exceeds GPT-4.1 and DeepSeek-R1 models on diverse table reasoning tasks. |
Parallel-SFT: Improving Zero-Shot Cross-Programming-Language Transfer for Code RL (2026.findings-acl)
Copied to clipboard
Zhaofeng Wu, Shiqi Wang, Boya Peng, Anuj Kumar Goyal, Melanie Kambadur, Sebastian Ruder, Yoon Kim, Chloe Bi
| Challenge: | Modern language models demonstrate impressive coding capabilities in common programming languages (PLs) but their performance in lower-resource PLs is often limited by training data availability. |
| Approach: | They propose a zero-shot cross-programming-language transfer task for code RL . they propose RL training in a source PL fails to improve performance on other target PLs . |
| Outcome: | The proposed approach improves transferability in Llama-3.1 code generation on parallel-stack model . it also improves performance on other target PLs, compared to single-PL SFT . |
Video-RTS: Rethinking Reinforcement Learning and Test-Time Scaling for Efficient and Enhanced Video Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Despite advances in reinforcement learning, data collection and fine-tuning remain costly and hard to scale. |
| Approach: | They propose a video-adaptive test-time scaling strategy that combines RL with a supervised fine-tuning strategy to improve video reasoning capability. |
| Outcome: | The proposed method surpasses existing models by 2.4% in accuracy using only 3.6% training samples. |
GFT: From Imitation to Reward Fine-Tuning with Unbiased Group Advantages and Dynamic Coefficient Rectification (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have demonstrated that supervised fine-tuning and reinforcement learning are effective in integrating knowledge injection with robust generalization. |
| Approach: | They propose a unified post-training framework that addresses intrinsic limitations of supervised fine-tuning and reinforcement learning. |
| Outcome: | The proposed framework surpasses SFT-based methods and yields policies that integrate more smoothly with subsequent RL training. |
Tailored Primitive Initialization is the Secret Key to Reinforcement Learning (2026.acl-long)
Copied to clipboard
| Challenge: | Reinforcement learning (RL) has emerged as a powerful paradigm for improving the reasoning capabilities of large language models. |
| Approach: | They propose a pipeline that automatically discovers thinking token patterns with reasoning primitives and curates SFT datasets to prepare LLMs for RL. |
| Outcome: | The proposed pipeline outperforms baseline methods on mathematical and logical reasoning benchmarks on RL tasks. |
VerIF: Verification Engineering for Reinforcement Learning in Instruction Following (2025.emnlp-main)
Copied to clipboard
| Challenge: | Best practices for RL in instruction following remain underexplored. |
| Approach: | They propose a verification method that combines rule-based code verification with LLM-based verification from a large reasoning model. |
| Outcome: | The proposed method achieves state-of-the-art performance among models of comparable size and generalizes well to unseen constraints. |
Training LLMs for Divide-and-Conquer Reasoning Elevates Test-Time Scalability (2026.acl-long)
Copied to clipboard
Xiao Liang, Zhong-Zhi Li, Zhenghao Lin, Eric Hanchen Jiang, Hengyuan Zhang, Yelong Shen, Kai-Wei Chang, Ying Nian Wu, Yeyun Gong, Weizhu Chen
| Challenge: | Large language models have demonstrated strong reasoning capabilities through step-by-step chain-of-thought (CoT) reasoning, but their strictly sequential nature constrains test-time scalability. |
| Approach: | They propose an end-to-end reinforcement learning framework to enhance LLMs' DAC-style reasoning capacity by decomposing a problem into subproblems and solving them sequentially. |
| Outcome: | The proposed model surpasses CoT by 8.6% and 6.3% on competition-level benchmarks and is available at the [github.com/MasterVito/DAC-RL]. |
SCALER: Synthetic Scalable Adaptive Learning Environment for Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Reinforcement learning (RL) is a principled way to enhance the reasoning capabilities of large language models, yet its effectiveness hinges on training signals that remain informative as models evolve. |
| Approach: | They propose a framework that sustains effective learning signals through adaptive environment design that transforms real-world programming problems into verifiable reasoning environments with controllable difficulty and unbounded instance generation. |
| Outcome: | The proposed framework outperforms baselines across diverse reasoning benchmarks and exhibits more stable, long-horizon training dynamics. |
TPS-Bench: Evaluating AI Agents’ Tool Planning & Scheduling Abilities in Compounding Tasks (2026.acl-long)
Copied to clipboard
| Challenge: | Large language model (LLM) agents have demonstrated strong problem-solving competence across domains like research and coding. |
| Approach: | They propose to use a tool repository to analyze the ability of large language model agents to solve complex problems. |
| Outcome: | The proposed model outperforms open-source and closed-source models in task completion rate and efficiency. |
MUSEG: Reinforcing Video Temporal Understanding via Timestamp-Aware Multi-Segment Grounding (2026.acl-long)
Copied to clipboard
Fuwen Luo, Shengfeng Lou, Chi Chen, Ziyue Wang, Chenliang Li, Weizhou Shen, Jiyue Guo, Peng Li, Ming Yan, Ji Zhang, Fei Huang, Yang Liu
| Challenge: | Existing methods for MLLMs struggle with fine-grained temporal reasoning . despite advances in video understanding, current methods struggle with time-sensitive tasks . |
| Approach: | They propose a time-stamp-aware multi-segment grounding method that enhances temporal understanding by introducing timestamps. |
| Outcome: | The proposed method outperforms existing methods on time-sensitive tasks and generalizes well across diverse temporal understanding scenarios. |
Mending the Holes: Mitigating Reward Hacking in Reinforcement Learning for Multilingual Translation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for training large language models rely heavily on high-quality parallel data, which are often scarce or unavailable for low-resource languages. |
| Approach: | They propose a reinforcement training method using only monolingual text to elevate LLMs’ translation capabilities on massive low-resource languages while retaining their performance on high-resourced languages. |
| Outcome: | The proposed model outperforms LLaMAX, one of the strongest open-source multilingual LLMs on 1,414 language directions on Flores-101 dataset. |
SPARKLE: A Structured and Plug-and-play Agentic Retrieval Policy for Adaptive RAG Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for integrating external knowledge rely on frozen large language models without explicit supervision or require costly LLM finetuning. |
| Approach: | They propose a structured and plug-and-play agentic retrieval policy with an additional proxy model to control the retrieval process. |
| Outcome: | Experiments on three in-domain and four out-of-domain QA benchmarks show that SPARKLE outperforms state-of the-art adaptive RAG models, achieving average improvements of 9.17% and 2.85%, respectively. |
MTP-RL: Acceleration of Reinforcement Learning Rollouts with Policy-Aligned Multi-Token Prediction (2026.findings-acl)
Copied to clipboard
| Challenge: | Reinforcement learning (RL) is widely applied to boost the performance of pretrained models, yet its training efficiency is severely constrained by rollout generation. |
| Approach: | They propose a framework that accelerates the rollout phase for diverse models by equipping a pipeline to equip the multi-layer parameter-sharing MTP for all models and an advantage-aware MTP optimization strategy. |
| Outcome: | The proposed framework achieves stable growth of acceptance length during RL training, and also accelerates RL rollouts, achieving an average 23.1%–55.3% reduction in rollout time compared to baselines. |
Robust Tool Use via Fission-GRPO: Learning to Recover from Execution Errors (2026.acl-long)
Copied to clipboard
Zhiwei Zhang, Fei Zhao, Rui Wang, Zezhong Wang, Bin Liang, Jiakang Wang, Yao Hu, Shaosheng Cao, Kam-Fai Wong
| Challenge: | Large language models (LLMs) can call tools effectively, but they remain brittle in multi-turn execution. |
| Approach: | They propose a framework that converts execution errors into on-policy corrective supervision within the RL training loop. |
| Outcome: | The proposed framework improves the error recovery rate of Qwen3-8B by 5.7% absolute and overall accuracy by 4.0% on BFCL v4 Multi-Turn. |
R-Search: Empowering LLM Reasoning with Search via Multi-Reward Reinforcement Learning (2026.findings-acl)
Copied to clipboard
Qingfei Zhao, Ruobing Wang, Dingling Xu, Daren Zha, Ma Bowen, Zhichun Wang, Shijie Jia, Limin Liu, Xin Wang
| Challenge: | Large language models (LLMs) have demonstrated impressive capabilities in multi-step and long-chain reasoning, but extending their reasoning capabilities to encompass deep interactions with search remains a non-trivial challenge. |
| Approach: | They propose a framework for Reasoning–Search integration that integrates multi-reward signals to optimize the reasoning–search interaction trajectories. |
| Outcome: | Experiments on seven datasets show that R-Search significantly outperforms mainstream RAG baselines. |
Sparse-RL: Breaking the Memory Wall in LLM Reinforcement Learning via Stable Sparse Rollouts (2026.acl-long)
Copied to clipboard
Sijia Luo, Xiaokang Zhang, Yuxuan Hu, Bohan Zhang, Ke Wang, Jinbo Su, Mengshu Sun, Lei Liang, Jing Zhang
| Challenge: | Existing methods for storing key-value caches during long-horizon rollouts cause performance collapses. |
| Approach: | They propose a new training paradigm that empowers stable RL training under sparse rollouts. |
| Outcome: | The proposed model reduces rollout overhead while maintaining the performance. |
Influence-based Online Experience Selection for Effective RLHF (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for RL fail to establish an interpretable connection between data and optimization objectives. |
| Approach: | They propose a data selection method that dynamically estimates the influence of individual training samples on policy optimization. |
| Outcome: | The proposed method significantly improves training effectiveness with fewer optimization steps. |